Real-time acquisition of high-resolution, wide field of view (FOV) and high dynamic range (HDR) images is essential for many military and civilian surveillance applications. For instance, there is an urgent need for an omnidirectional imaging system in many surveillance applications where the system, with sufficient resolution and frame rate, can monitor the activities in all directions simultaneously across a very large operating field (e.g. spherical or complimentary hemispherical coverage) while being able to rapidly zoom into one or multiple objects of interest for reliable identification and characterization of the objects. Such a sensor needs to provide both excellent situational awareness and adequate detail resolvability. This type of sensors, if available, can find myriads of applications in both military and commercial markets.
However, when designing an optical imaging system, finite sensor resolution and data bandwidth impose limits on the spatial resolution and FOV achievable in state-of-the-art imaging systems. There is a well-known inherent tradeoff between the FOV and the resolving power for most conventional imaging techniques with a fixed number of pixels: the wider the FOV, the lower the resolving power. Using the traditional cluster-based omnidirectional cameras as an example, in order to achieve a 1 arc minute (−300 micro-rad) angular resolution, it requires at least 50 small FOV cameras (e.g. FOV: 33°×25°) with a 5-Mega pixel sensor on each to cover a spherical field of 360°×360°, which results in a minimum of 250 Mega pixels to be captured, stored and transmitted for a single spherical panoramic image, barring any pixel loss and FOV overlap. To achieve an angular resolution of 2 arc seconds requires a prohibitive number of cameras in the order of thousands to cover a spherical field. As a result, the cost and size of a camera-cluster-based system will be unacceptable for many surveillance applications, not mentioning that clustering over thousands of high-resolution cameras imposes great challenges to the state-of-the-art data management and image processing technologies.
Foveation techniques can actively track and capture a region of interest with high resolution sensor without losing the imaging capability of the peripheral area, similar to the foveation properties of the human vision system. Various imaging systems have been developed to explore the potential of applying the foveation technique in imaging applications. For example, Sandini et al. developed a retina-like CMOS sensor with spatially-variant resolution to mimic the human retina (G. Sandini, P. Questa, D. Scheffer and A. Mannucci, “A Retina-like CMOS sensor and its applications,” Proceedings of IEEE Workshop on Sensor Array and Multichannel Signal Process. (2000), pp. 514-9). Martinez and Wick proposed to use a liquid crystal spatial light modulator to dynamically correct the aberrations at the foveated region inside a wide FOV of imaging system (T. Martinez, D. V. Wick and S. R. Restaino, “Foveated, wide field-of-view imaging system using a liquid crystal spatial light modulator,” Opt. Express 8, 555-60 (2001); D. V. Wick, T. Martinez, S. R. Restaino and B. R. Stone, “Foveated imaging demonstration,” Opt. Express 10, 60-5 (2002)). The aforementioned approaches use only single-sensor to capture both the peripheral region and the foveated region which limits the overall information throughput of the system. Alternatively, Hua and Liu proposed a dual-sensor approach to the foveation imaging technology where two separate sensors are used to capture the peripheral region and the foveated region (Hong Hua and Sheng Liu, “Dual-Sensor foveated imaging system,” APPLIED OPTICS, Vol. 47, No. 3, 317-327, 2008). Comparing with the single sensor approach, the dual sensor approach uses two different sensors which can be in different size and different resolution, which has potential to yield high information throughput with low-cost detectors. The main dis-advantage of the dual-sensor approach is that the system employs an a focal system structure which usually has a limited ability to achieve large peripheral FOV and often results in a bulky system.